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JEFFERY, Administrative Patent Judge. 

DECISION ON APPEAL 
1 Appellant appeals under 35 U.S.C. § 134 from the Examiner's 
rejection of claims 1-20. We have jurisdiction under 35 U.S.C. § 6(b). We 
affirm. 
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STATEMENT OF THE CASE 
Appellant invented a method for tracking target objects in images 
provided from a non- stationary camera that employs motion estimation and 
compensation techniques. Specifically, low resolution images are computed 
from two consecutive image frames. Feature points are determined and 
matched between the two low resolution images. Statistical techniques 
determine a homography matrix describing motion between the 
corresponding feature points in the original images, and this matrix is used 
to align the original images. Differences between aligned images are 
identified to indicate movement of one or more objects in the image. 1 Claim 
1 is illustrative: 

1 . A method of aligning a first image to a second image, comprising: 

determining a first alignment approximation, based on distances 
between one or more points in the first image and the second image, with the 
first and second images at a first resolution, 

aligning the second image to the first image, based on the first 
alignment approximation, to form an initially aligned second image, 

determining a second alignment approximation, based on the 
distances between one or more points in the first image and the initially 
aligned second image, with the first and second images at a second 
resolution different from the first resolution, and 

aligning the second image to the first image, based on the combination 
of the first and second alignment approximations. 

The Examiner relies on the following prior art references to show 
unpatentability: 
1 See generally Spec. 3:1-15. 
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Frazier US 5,651,075 Jul. 22, 1997 

Gupta US 5,848,121 Dec. 8, 1998 

Claims 1-20 stand rejected under 35 U.S.C. § 103(a) as unpatentable 
over Gupta and Frazier. 

Rather than repeat the arguments of Appellant or the Examiner, we 
refer to the Briefs and the Answers 2 for their respective details. In this 
decision, we have considered only those arguments actually made by 
Appellant. Arguments which Appellant could have made but did not make 
in the Brief have not been considered and are deemed to be waived. See 37 
C.F.R. §41.37(c)(l)(vii). 

OPINION 
Independent Claim 1 

In rejecting claims under 35 U.S.C. § 103, it is incumbent upon the 
Examiner to establish a factual basis to support the legal conclusion of 
obviousness. See In re Fine, 837 F.2d 1071, 1073 (Fed. Cir. 1988). In so 
doing, the Examiner must make the factual determinations set forth in 
Graham v. John Deere Co., 383 U.S. 1,17 (1966). 

Discussing the question of obviousness of a patent that claims a 

combination of known elements, KSR Int'l v. Teleflex, Inc., Ill S. Ct. 1727 

(2007), explains: 

When a work is available in one field of endeavor, design 
incentives and other market forces can prompt variations of it, 
either in the same field or a different one. If a person of 
ordinary skill can implement a predictable variation, §103 

2 We refer to the Answer mailed April 18, 2005 and the Supplemental 
Answer mailed October 4, 2005. 
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likely bars its patentability. For the same reason, if a technique 
has been used to improve one device, and a person of ordinary 
skill in the art would recognize that it would improve similar 
devices in the same way, using the technique is obvious unless 
its actual application is beyond his or her skill. Sakraida [v. AG 
Pro, Inc., 425 U.S. 273 (1976)] and Anderson' s-Black Rock[, 
Inc. v. Pavement Salvage Co., 396 U.S. 57 (1969)] are 
illustrative — a court must ask whether the improvement is more 
than the predictable use of prior art elements according to their 
established functions. 

KSR, 127 S. Ct. at 1740. If the claimed subject matter cannot be fairly 

characterized as involving the simple substitution of one known element for 

another or the mere application of a known technique to a piece of prior art 

ready for the improvement, a holding of obviousness can be based on a 

showing that "there was an apparent reason to combine the known elements 

in the fashion claimed." Id. at 1740-41. Such a showing requires "some 

articulated reasoning with some rational underpinning to support the legal 

conclusion of obviousness. . . . [H]owever, the analysis need not seek out 

precise teachings directed to the specific subject matter of the challenged 

claim, for a court can take account of the inferences and creative steps that a 

person of ordinary skill in the art would employ." Id. at 1741 (quoting In re 

Kahn, 441 F.3d 977, 988 (Fed. Cir. 2006)). 

If the Examiner's burden is met, the burden then shifts to the 
Appellant to overcome the prima facie case with argument and/or evidence. 
Obviousness is then determined on the basis of the evidence as a whole and 
the relative persuasiveness of the arguments. See In re Oetiker, 977 F.2d 
1443, 1445 (Fed. Cir. 1992). 

Regarding independent claim 1, the Examiner's rejection essentially 
finds that Gupta teaches an image alignment method with every claimed 
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feature but does not explicitly state that images are aligned based on the 
combination of first and second image approximations. Although the 
Examiner assumes that Gupta combines the opacified and mask images, the 
Examiner nevertheless cites Frazier as teaching combining image 
approximations by applying a Laplacian operator, and concludes that it 
would have been obvious to one of ordinary skill in the art at the time of the 
invention to incorporate this teaching into Gupta's system (Ans. 14-15). 

Appellant argues that Gupta does not teach or suggest, among other 
things, determining the first and second alignment approximations based on 
the recited distances between points in the first and second images as claimed. 
Rather, Appellant contends, Gupta determines match points within the 
mask and opacified images. Then, an image-to-image transform is 
generated that is repeatedly used. Appellant adds that Frazier's processing 
(edge enhancement and shadow reduction) is performed on a single image, but 
the reference does not teach manipulating, enhancing, or aligning multiple 
images (App. Br. 5-10; Reply Br. 2-7). 

The Examiner notes that the recited first and second alignment 
approximations are equivalent to Gupta's match point generation technique 
which proceeds hierarchically from the lowest resolution to a highest 
resolution. According to the Examiner, these alignment approximations 
are based on distances between one or more points in the mask and opacified 
images (Ans. 3, 5; Supp. Ans. 1-3). 

The issue before us, then, is whether the collective teachings of Gupta 
and Frazier teach or suggest the limitations of independent claim 1 , 
namely the first and second alignment approximation determinations and 
image alignment steps. For the following reasons, we find that the 
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collective teachings of the references would have reasonably suggested these 
limitations. 

Gupta discloses a method for digital subtraction angiography (DSA) — 
a known procedure for observing vasculature. In a known DSA method, 
X-ray images are taken before and after an X-ray contrast agent is injected 
into the blood vessels to produce the mask and opacified images, respectively. 
Subtracting the mask image from the opacified image should remove all but the 
image data associated with the opacified blood vessels, but nonetheless can 
produce artifacts due to, among other things, patient motions and hysteresis 
(Gupta, col. 1,1. 10 -col. 2, 1. 15). 

To minimize these artifacts, Gupta's method obtains sub-pixel 
registration of the mask and opacified images by first implementing a match 
point generation technique 14 as shown in Figure 1 (Step 14). This technique 
derives a set of two-dimensional points in the mask image and their 
corresponding points in the opacified image (Gupta, col. 3, 11. 1-4). Following 
match point generation, locally-adaptive image-to-image warp generation 16 is 
performed in which the matched points in the mask image are mapped to 
corresponding points in the opacified image (Gupta, col. 3, 11. 1 1-19; Fig. 1 
(Step 16)). Then, the logarithm of each pixel value in the transformed (warped) 
image is subtracted from the logarithm of the corresponding pixel values in 
opacified image to produce the resulting DSA image (Gupta, col. 3, 11. 20-26; 
Fig. 1 (Steps 18 and 20)). 

The match point generation technique 14 is further detailed in Figure 2. 
First, an image hierarchy of various resolutions is created to accelerate the 
generation of match points (Gupta, col. 3, 11. 35-42; Fig. 2, Step 54). The 
matching process then begins with the lowest resolution image and progresses 
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to the highest resolution image (Gupta, col. 3, 11. 44-46). To this end, a small 
tile of imagery around a point in the mask image is correlated with all tiles in the 
opacified image - a technique that proceeds hierarchically from the lowest to 
highest resolution images. The center of the tile in the opacified image that 
gives the maximum correlation is identified as the corresponding match point 
(Gupta, col. 3, 11. 52-62). 

Once a set of match points is computed, an image-to-image transform is 
generated for registering the image tiles between the mask and opacified images 
(Gupta, col. 4, 11. 7-10; Fig. 2, Step 60). The matching process is then repeated 
using this new transform, ultimately resulting in a grid of match points in the 
mask image and their corresponding match points in the opacified image 
(Gupta, col. 4, 11. 7-18; Fig. 2, Step 62). 

Then, a suitable displacement (dx, dy) is determined by interpolation that 
is added to each grid point in the mask image to find the coordinates of the 
corresponding point in the opacified image. Thus, each pixel's location in the 
mask image is transformed by interpolation to find the location of the 
corresponding pixel in the opacified image. Then, the intensity of each pixel in 
the mask image is log-subtracted from the corresponding pixel in the opacified 
image (Gupta, col. 4, 11. 19-46). 

Based on this functionality, we agree with the Examiner (Ans. 3; Supp. 
Ans. 1-2) that the repetitive matching process in Gupta's match point generation 
technique that proceeds from lower to higher resolutions reasonably 
corresponds to the recited first and second alignment approximations, 
respectively. In our view, correlating particular areas of the mask image with 
corresponding areas of the opacified image reasonably constitutes an "alignment 
approximation" when the term is given its broadest reasonable interpretation. 
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Furthermore, we find that these alignment approximations are based, at 
least in part, on distances between corresponding points in the respective 
images. In this regard, we note that Gupta expressly states that the image tiles 
in the mask and opacified images may be rotated or translated with respect to 
each other - a mismatch that is corrected by a transformation based on user- 
provided match points (Gupta, col. 3, 11. 63 - col. 4, 1. 2; emphasis added). In 
view of this displacement between the corresponding areas in the mask and 
opacified images - a displacement that would clearly involve distances between 
these corresponding points - the correlation between these corresponding areas 
in Gupta's match point generation technique (i.e., the "first alignment 
approximation") would therefore be based on, at least in part, this displacement. 

We also find that the image-to-image transform generated based on the 
determined match points (i.e., Step 60 in Fig. 2) reasonably corresponds to the 
recited alignment steps. Significantly, Gupta indicates that the image-to-image 
transform is for registering the image tiles between the mask and opacified 
images (Gupta, col. 4, 11. 7-10). The term "register" is defined, in pertinent part, 
as "[t]o adjust so as to be properly aligned" and, alternatively, "[t]o be in proper 
alignment." 3 Based on these facts, skilled artisans would readily understand that 



3 The American Heritage Dictionary of the English Language, 4th ed., 2000, 
available at http://www.bartleby.com/61/55/R0125500 (last visited Feb. 21, 
2008). Although Appellant contends that the terms "aligned," "aligning," 
and "alignment" are "clearly defined" in the Specification (App. Br. 9-10), 
we do not find an express definition in the Specification tantamount to a 
special definition of the terms. Accordingly, absent a specific definition, we 
construe the terms with their plain meaning (i.e., the ordinary and customary 
meaning given to the terms by those of ordinary skill in the art). See Brookhill- 
Wilk 1, LLC. v. Intuitive Surgical, Inc., 334 F.3d 1294, 1298 (Fed. Cir. 2003); 
see also Phillips v. AWHCorp., 415 F.3d 1303, 1312 (Fed. Cir. 2005) (en 
banc). 
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the image-to-image transform in Gupta effectively provides an alignment 
function at least with respect to the corresponding image tiles in the mask and 
opacified images. Therefore, the image-to-image transform created at a lower 
resolution reasonably corresponds to "aligning the second image to the first 
image" at least in part - an alignment that is "based on" the generated match 
points that correspond to the recited first alignment approximation. Likewise, 
the alignment approximation and image-to-image transform determined at the 
higher resolution (i.e., following the lower resolution determinations) 
reasonably corresponds to the second alignment approximation and alignment 
steps. 

Although we find the Examiner's reliance on Frazier problematic for 
teaching that the second alignment step is based on the combination of the first 
and second alignment approximations (Ans. 4, 5, 15; Supp. Ans. 3) essentially 
for the reasons indicated by Appellant (App. Br. 6-10; Reply Br. 4, 6, 7), 4 we 
nonetheless find that Gupta reasonably teaches this limitation. In our view, the 
iterative, sequential generation of image-to-image transforms in Gupta's match 
point generation technique (i.e., Steps 56-62 of Figure 2) effectively results in 



4 However, we note in passing that Appellant's contention that the 
Examiner's reliance on various aspects of Frazier as allegedly constituting 
an improper new grounds of rejection is not an appealable matter, but rather 
a petitionable matter, and is therefore not before us. See MPEP § 
1207.03(IV) (noting that if Appellant believes that an Examiner's Answer 
contains a new ground of rejection not identified as such, Appellant may file 
a petition under 37 C.F.R. § 1.181(a) within two months from the mailing of 
the Examiner's Answer); see also MPEP § 706.01 ("[T]he Board will not 
hear or decide issues pertaining to objections and formal matters which are 
not properly before the Board."); MPEP § 1201 ("The Board will not 
ordinarily hear a question that should be decided by the Director on 
petition...."). 



9 



Appeal 2007-3332 
Application 09/854,119 

multiple sequential alignments that are based, at least in part, on the 
corresponding determined match points. Significantly, these sequential 
determinations are iterative. Thus, image-to-image transforms that are 
iteratively determined at higher resolutions would be based on, at least in part, 
the current and previous match point determinations (i.e., the combination of 
alignment approximations determined at higher and lower resolutions). 

For the foregoing reasons, we find that Appellant has not persuasively 
rebutted the Examiner's prima facie case of obviousness based on the collective 
teachings of Gupta and Frazier. Accordingly, the Examiner's rejection of that 
claim is sustained. 

Claims 2 and 3 

We will also sustain the Examiner's rejection of claim 2, calling for 
aligning the initially aligned second image based on the second alignment 
approximation, essentially for the reasons indicated in connection with claim 1 
above. We therefore incorporate that discussion here by reference. 

Regarding claim 3, we will also sustain the Examiner's rejection of that 
claim for similar reasons. Furthermore, we disagree with Appellant's 
contention (App. Br. 12) that the opacified image in Gupta only has a single 
resolution as compared to the multiple mask images. On the contrary, Gupta 
expressly states that when the image hierarchy is created in connection with 
match point generation (i.e., Step 54 in Figure 2), both images (i.e., the mask 
and opacified images) are successively reduced to half their size and resolution 
via subsampling (Gupta, col. 3, 11. 39-42; col. 6, 11. 1-5 (text of claim 4)). 



10 



Appeal 2007-3332 
Application 09/854,119 

For the foregoing reasons, Appellant has not persuasively rebutted the 
Examiner's prima facie case of obviousness for claims 2 and 3. The 
Examiner's rejection of those claims is therefore sustained. 

Claim 4 

We will also sustain the Examiner's rejection of claim 4 calling for 
applying the RANSAC algorithm. We agree with the Examiner (Ans. 7; Supp. 
Ans. 5-6) that such algorithms are well known - a fact evidenced by 
Appellant's own Specification. 5 In our view, using such well-known algorithms 
that identify and ignore inconsistent points in a set in the image processing 
system of Gupta would have been tantamount to the predictable use of prior art 
elements according to their established functions - an obvious improvement. 
See KSR, 127 S. Ct. at 1740. The rejection is therefore sustained. 

Claim 5 

We will also sustain the Examiner's rejection of claim 5 essentially for 
the reasons indicated in connection with claim 1 . As we indicated previously, 
Gupta expressly states that the image tiles in the mask and opacified images 
may be rotated or translated with respect to each other - a mismatch that is 
corrected by a transformation based on user-provided match points (Gupta, col. 
3, 11. 63 - col. 4, 1. 2; emphasis added). This feature, in our view, amply 
suggests that the rotation and translation components are approximated at least 
to correct the mismatches resulting therefrom. For the foregoing reasons, 

5 Appellant's Specification states that "[t]he RANSAC algorithm, common 
in the art,., .identifies and ignores 'outliers', points in a set of sample point 
[sic] that are inconsistent with most of the other points in the set" (Spec. 
6:23-25; emphasis added). 
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Appellant has not persuasively rebutted the Examiner's prima facie case of 
obviousness for claim 5 and the rejection is therefore sustained. 

Claims 6 and 7 

We will also sustain the Examiner's rejection of claims 6 and 7 which 
call for, in pertinent part, a 3x3 nomographic matrix. The Examiner indicates 
that methods involving nomographic matrices are well known in the art (Ans. 8; 
Supp. Ans. 6). Appellant does not dispute this position, but rather argues that 
the Examiner has simply restated functions performed by the cited references 
and has not shown how a 3x3 homographic matrix would be used in the cited 
references to determine alignment approximations (App. Br. 15-16; Reply Br. 
9-10). 

Based on the record before us, we see no reason why skilled artisans 
could not have employed such a matrix to perform the alignment 
approximations in Gupta. Clearly, Gupta's match point generation technique 
uses multiple coordinates associated with the respective points in the respective 
images. In our view, to use known homographic matrix techniques to process 
data and render the calculations associated with these points in Gupta's image 
processing system would have been tantamount to the predictable use of prior 
art elements according to their established functions. See KSR, 127 S. Ct. at 
1740. We will therefore sustain the Examiner's rejection of claims 6 and 7. 

Claim 8 

We will also sustain the Examiner's rejection of claim 8. Although 
Gupta does not expressly indicate that corners are identified based on a 
determination of Minimum Intensity Changes as claimed, Gupta does state that 
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a feature-based matching scheme could be used in lieu of a correlation-based 
scheme (Gupta, col. 4, 11. 56-58; emphasis added). Furthermore, Appellant's 
own Specification admits that Minimum Intensity Change (MIC) corner 
detectors are well known in the art (Spec. 8: 11-19). 

Based on these teachings, we see no reason why skilled artisans could not 
have utilized a conventional feature-based scheme, such as MIC corner 
detection, in Gupta. In view of Gupta's teaching of using feature-based 
matching schemes, incorporating a conventional corner detection scheme in 
Gupta is tantamount to the predictable use of prior art elements according to 
their established functions — an obvious improvement. See KSR, 127 S. Ct. at 
1 740. We will therefore sustain the Examiner's rejection of claim 8. 

Claims 9, 12, 13, 15, and 18-20 
We will also sustain the Examiner's rejection of independent claim 9 
calling for, among other things, tracking an object based on first and second 
images by detecting motion by comparing the set of aligned images. At the 
outset, we note that our previous discussion pertaining to claim 1 applies 
equally here and we incorporate that discussion by reference. Furthermore, we 
agree with the Examiner (Ans. 9) that Gupta, in effect, detects motion as 
claimed. In connection with match point generation, Gupta states that a grid of 
points in the mask image is square, but the grid in the opacified image is not 
square due to, among other things, patient motion (Gupta, col. 4, 11. 14-18). 

The clear import of this discussion is that comparing the respective 
shapes of the grids associated with the mask and opacified images in Gupta can 
determine whether patient motion exists - and therefore detect such motion. 
That is, if the grids in the mask and opacified images were both square, then no 
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motion would be detected. But if the grids had different shapes, then the system 
would detect motion, hysteresis, or other such effects. Although we recognize 
that motion would not necessarily be detected by this comparison due to the 
possible existence of hysteresis and other such effects, Gupta nevertheless 
strongly suggests that patient motion would be detected by the system if it were 
present. 

Regarding claims 12 and 18, we add that the location of the points in the 
grids in the mask and opacified images is also determined, and that such a 
determination effectively determines a location of the object (i.e., that part of the 
patient). Comparing the respective grids' shape in Gupta as noted above thus 
involves comparing the location of the object in each image as claimed. 

For the foregoing reasons, we find that Gupta's system, in effect, tracks 
an object (the patient) and detects motion by comparing aligned images as 
claimed. The Examiner's rejections of claims 9 and 12 are therefore sustained. 
Likewise, we will sustain the Examiner's rejection of independent claim 13 for 
similar reasons as claim 13 recites commensurate limitations. Since Appellant 
has not separately argued the patentability of dependent claim 15, it falls 
with independent claim 13. See 37 C.F.R. § 41.37(c)(l)(vii). 

Regarding claims 19 and 20, our previous discussion pertaining to 
claims 5-7 above applies equally here and we incorporate that discussion by 
reference. 

Claims 10 and 14 

We will also sustain the Examiner's rejection of claims 10 and 14 for the 
same reasons as we indicated with respect to claim 3 above, and we therefore 
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incorporate that discussion by reference. The rejections of claims 10 and 14 are 
therefore sustained. 

Claim 11 

We will also sustain the Examiner's rejection of claim 1 1. At the outset, 
we reiterate that Gupta teaches that a feature-based matching scheme could be 
used in lieu of a correlation-based scheme (Gupta, col. 4, 11. 56-58; emphasis 
added). In our view, skilled artisans would readily recognize that such features 
could include color. 

Although Gupta does not say what colors are present in the X-ray images 
processed by that system, we agree with the Examiner (Ans. 10) that skilled 
artisans would recognize that typical X-ray images contain at least gray, black, 
and white. Appellant's argument that gray, black, and white are not colors 
(App. Br. 20-21 ; Reply Br. 11-12) is unavailing. Although black and white are 
not within the visible color spectrum (i.e., they are not elementary colors), black 
and white nonetheless comprise colors. Specifically, white can be produced by 
combining all three primary "additive colors" (red, green, and blue-violet). 
Likewise, mixing three primary "subtractive colors" (magenta, yellow, and 
cyan (blue-green)) yields black. 6 And even if gray is simply a mixture of 
black and white as Appellant argues, skilled artisans would readily recognize 
that gray would effectively constitute a mixture of the additive and 
subtractive colors of both white and black as noted above. 

Therefore, even assuming that X-ray images contained solely white, 
black, and gray and identification was based solely on matching these 

6 See "Color," in The Columbia Encyclopedia, 6th ed., available at 
http://www.bartleby.com/65/co/color.html (last visited Feb. 21, 2008). 
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pigments, such matching would still fully meet "color matching" as white, 
black, and gray each comprise colors. In any event, we add that extending 
the ability to match white, black, and gray to matching other colors would 
have been well within the level of skilled artisans. "A person of ordinary 
skill is also a person of ordinary creativity, not an automaton." See KSR, 
127 S. Ct. at 1742. 

For the foregoing reasons, we will sustain the Examiner's rejection of 
claim 1 1 . 

Claim 16 

We will also sustain the Examiner's rejection of claim 16. We agree 
with the Examiner (Ans. 11-12) that Gupta's system must have at least a 
temporary memory for storing a representation of a target image. We reach 
this conclusion noting that the purpose of Gupta is to ultimately render a 
DSA image with reduced artifacts by subtracting the transformed mask 
image from the opacified image. Further, nothing in the claim precludes the 
identified "target" to constitute the pixel intensity values of the mask image 
that are log- subtracted from the corresponding pixels in the opacified image. 
This target identification would be based, at least in part, on a 
"representation of a target image" in the form of a stored log- subtraction 
algorithm or process and the values used by such a process. 

For the foregoing reasons, we will sustain the Examiner's rejection of 
claim 16. 

Claim 17 
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We will also sustain the Examiner's rejection of claim 17 for the same 
reasons as we indicated previous in connection with claim 1 1 . We therefore 
incorporate that discussion here by reference. 

DECISION 

We have sustained the Examiner's rejection with respect to all claims 
on appeal. Therefore, the Examiner's decision rejecting claims 1-20 is 
affirmed. 

No time period for taking any subsequent action in connection with 
this appeal may be extended under 37 C.F.R. § 1.136(a)(l)(iv). 

AFFIRMED 
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